Import¶

Before running the following cells you must install requirements.txt with: 'pip install -r requirements.txt'

In [ ]:
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import plotly.express as px
import time

color_pal = plt.rcParams['axes.prop_cycle'].by_key()['color']
from fredapi import Fred

from dotenv import load_dotenv
import os

plt.style.use('fivethirtyeight')
pd.set_option('display.max_columns', 500)

from datetime import datetime, timedelta
import calendar

import plotly.io as pio 
pio.renderers.default = 'notebook'

To acces Fred API you must register and obtain your own personal and unique key from https://fred.stlouisfed.org/docs/api/api_key.html

Then you have to create an '.env' file in your directory so you can import fred_key using dotenv.

In [ ]:
fred_api_key = 'put_your_key_here'
# load env variables 
load_dotenv() 
fred_api_key = os.getenv("fred_api_key")
print(fred_api_key)
4fc356a5003ef269ea9234d9df71576f

1. Fred Object¶

In [ ]:
fred = Fred(api_key=fred_api_key)

1.1 Search data¶

In [ ]:
sp_search = fred.search('S&P', limit=1000, order_by='popularity')
sp_search.head()
Out[ ]:
id realtime_start realtime_end title observation_start observation_end frequency frequency_short units units_short seasonal_adjustment seasonal_adjustment_short last_updated popularity notes
series id
BAMLH0A0HYM2 BAMLH0A0HYM2 2024-05-01 2024-05-01 ICE BofA US High Yield Index Option-Adjusted S... 1996-12-31 2024-04-30 Daily, Close D Percent % Not Seasonally Adjusted NSA 2024-05-01 09:23:14-05:00 92 The ICE BofA Option-Adjusted Spreads (OASs) ar...
CSUSHPINSA CSUSHPINSA 2024-05-01 2024-05-01 S&P CoreLogic Case-Shiller U.S. National Home ... 1987-01-01 2024-02-01 Monthly M Index Jan 2000=100 Index Jan 2000=100 Not Seasonally Adjusted NSA 2024-04-30 08:12:01-05:00 88 For more information regarding the index, plea...
SP500 SP500 2024-05-01 2024-05-01 S&P 500 2014-05-02 2024-05-01 Daily, Close D Index Index Not Seasonally Adjusted NSA 2024-05-01 19:10:58-05:00 83 The observations for the S&P 500 represent the...
MEHOINUSA672N MEHOINUSA672N 2024-05-01 2024-05-01 Real Median Household Income in the United States 1984-01-01 2022-01-01 Annual A 2022 CPI-U-RS Adjusted Dollars 2022 CPI-U-RS Adjusted $ Not Seasonally Adjusted NSA 2023-09-12 13:48:02-05:00 83 Household data are collected as of March. As ...
BAMLH0A0HYM2EY BAMLH0A0HYM2EY 2024-05-01 2024-05-01 ICE BofA US High Yield Index Effective Yield 1996-12-31 2024-04-30 Daily, Close D Percent % Not Seasonally Adjusted NSA 2024-05-01 09:23:13-05:00 82 This data represents the effective yield of th...

2. Pull raw data & plot¶

In [ ]:
sp500 = fred.get_series(series_id='SP500')
sp500.plot(figsize=(10,5), title='S&P 500', lw=1)
Out[ ]:
<Axes: title={'center': 'S&P 500'}>
No description has been provided for this image

3. Pull and join multiple data series¶

In [ ]:
uemp_show = fred.search('unemployment', limit=1000)
uemp_show.head()
Out[ ]:
id realtime_start realtime_end title observation_start observation_end frequency frequency_short units units_short seasonal_adjustment seasonal_adjustment_short last_updated popularity notes
series id
UNRATE UNRATE 2024-05-01 2024-05-01 Unemployment Rate 1948-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-05 07:50:02-05:00 95 The unemployment rate represents the number of...
UNRATENSA UNRATENSA 2024-05-01 2024-05-01 Unemployment Rate 1948-01-01 2024-03-01 Monthly M Percent % Not Seasonally Adjusted NSA 2024-04-05 07:50:24-05:00 58 The unemployment rate represents the number of...
UNEMPLOY UNEMPLOY 2024-05-01 2024-05-01 Unemployment Level 1948-01-01 2024-03-01 Monthly M Thousands of Persons Thous. of Persons Seasonally Adjusted SA 2024-04-05 07:50:18-05:00 71 The series comes from the 'Current Population ...
NROU NROU 2024-05-01 2024-05-01 Noncyclical Rate of Unemployment 1949-01-01 2034-10-01 Quarterly Q Percent % Not Seasonally Adjusted NSA 2024-02-08 14:46:04-06:00 69 Starting with the July, 2021 report: An Update...
CCSA CCSA 2024-05-01 2024-05-01 Continued Claims (Insured Unemployment) 1967-01-07 2024-04-13 Weekly, Ending Saturday W Number Number Seasonally Adjusted SA 2024-04-25 07:48:04-05:00 74 Continued claims, also referred to as insured ...
In [ ]:
unrate = fred.get_series('UNRATE')
unrate.plot(figsize=(10,5), title='Unemployment UNRATE', lw=1)
Out[ ]:
<Axes: title={'center': 'Unemployment UNRATE'}>
No description has been provided for this image
In [ ]:
unemp_df = fred.search('unemployment rate state', filter=('frequency', 'Monthly'))
unemp_df.head()
Out[ ]:
id realtime_start realtime_end title observation_start observation_end frequency frequency_short units units_short seasonal_adjustment seasonal_adjustment_short last_updated popularity notes
series id
UNRATE UNRATE 2024-05-01 2024-05-01 Unemployment Rate 1948-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-05 07:50:02-05:00 95 The unemployment rate represents the number of...
UNRATENSA UNRATENSA 2024-05-01 2024-05-01 Unemployment Rate 1948-01-01 2024-03-01 Monthly M Percent % Not Seasonally Adjusted NSA 2024-04-05 07:50:24-05:00 58 The unemployment rate represents the number of...
LNS14000006 LNS14000006 2024-05-01 2024-05-01 Unemployment Rate - Black or African American 1972-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-05 07:50:16-05:00 72 The series comes from the 'Current Population ...
UNEMPLOY UNEMPLOY 2024-05-01 2024-05-01 Unemployment Level 1948-01-01 2024-03-01 Monthly M Thousands of Persons Thous. of Persons Seasonally Adjusted SA 2024-04-05 07:50:18-05:00 71 The series comes from the 'Current Population ...
LNU04000006 LNU04000006 2024-05-01 2024-05-01 Unemployment Rate - Black or African American 1972-01-01 2024-03-01 Monthly M Percent % Not Seasonally Adjusted NSA 2024-04-05 07:50:55-05:00 23 The series comes from the 'Current Population ...
In [ ]:
unemp_df = unemp_df.query('seasonal_adjustment == "Seasonally Adjusted" and units == "Percent"')
unemp_df = unemp_df.loc[unemp_df['title'].str.contains('Unemployment Rate in')]
unemp_df
Out[ ]:
id realtime_start realtime_end title observation_start observation_end frequency frequency_short units units_short seasonal_adjustment seasonal_adjustment_short last_updated popularity notes
series id
CAUR CAUR 2024-05-01 2024-05-01 Unemployment Rate in California 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:02-05:00 64 None
TXUR TXUR 2024-05-01 2024-05-01 Unemployment Rate in Texas 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:03-05:00 56 None
NYUR NYUR 2024-05-01 2024-05-01 Unemployment Rate in New York 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:03-05:00 53 None
FLUR FLUR 2024-05-01 2024-05-01 Unemployment Rate in Florida 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:03-05:00 51 None
PAUR PAUR 2024-05-01 2024-05-01 Unemployment Rate in Pennsylvania 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:04-05:00 44 None
OHUR OHUR 2024-05-01 2024-05-01 Unemployment Rate in Ohio 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:04-05:00 41 None
MIUR MIUR 2024-05-01 2024-05-01 Unemployment Rate in Michigan 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:04-05:00 38 None
ALUR ALUR 2024-05-01 2024-05-01 Unemployment Rate in Alabama 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:04-05:00 38 None
MAUR MAUR 2024-05-01 2024-05-01 Unemployment Rate in Massachusetts 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:05-05:00 37 None
NCUR NCUR 2024-05-01 2024-05-01 Unemployment Rate in North Carolina 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:06-05:00 34 None
ILUR ILUR 2024-05-01 2024-05-01 Unemployment Rate in Illinois 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:07-05:00 31 None
NJUR NJUR 2024-05-01 2024-05-01 Unemployment Rate in New Jersey 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:11-05:00 22 None
GAUR GAUR 2024-05-01 2024-05-01 Unemployment Rate in Georgia 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:05-05:00 34 None
COUR COUR 2024-05-01 2024-05-01 Unemployment Rate in Colorado 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:05-05:00 34 None
AZUR AZUR 2024-05-01 2024-05-01 Unemployment Rate in Arizona 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:05-05:00 33 None
WIUR WIUR 2024-05-01 2024-05-01 Unemployment Rate in Wisconsin 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:08-05:00 33 None
VAUR VAUR 2024-05-01 2024-05-01 Unemployment Rate in Virginia 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:06-05:00 31 None
KYUR KYUR 2024-05-01 2024-05-01 Unemployment Rate in Kentucky 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:13-05:00 20 None
SCUR SCUR 2024-05-01 2024-05-01 Unemployment Rate in South Carolina 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:06-05:00 30 None
NMUR NMUR 2024-05-01 2024-05-01 Unemployment Rate in New Mexico 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:09-05:00 28 None
MNUR MNUR 2024-05-01 2024-05-01 Unemployment Rate in Minnesota 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:08-05:00 27 None
TNUR TNUR 2024-05-01 2024-05-01 Unemployment Rate in Tennessee 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:07-05:00 29 None
WAUR WAUR 2024-05-01 2024-05-01 Unemployment Rate in Washington 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:07-05:00 29 None
MDUR MDUR 2024-05-01 2024-05-01 Unemployment Rate in Maryland 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:09-05:00 28 None
ORUR ORUR 2024-05-01 2024-05-01 Unemployment Rate in Oregon 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:07-05:00 28 None
WVUR WVUR 2024-05-01 2024-05-01 Unemployment Rate in West Virginia 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:10-05:00 28 None
NVUR NVUR 2024-05-01 2024-05-01 Unemployment Rate in Nevada 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:10-05:00 27 None
ARUR ARUR 2024-05-01 2024-05-01 Unemployment Rate in Arkansas 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:11-05:00 27 None
OKUR OKUR 2024-05-01 2024-05-01 Unemployment Rate in Oklahoma 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:11-05:00 27 None
MOUR MOUR 2024-05-01 2024-05-01 Unemployment Rate in Missouri 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:10-05:00 26 None
AKUR AKUR 2024-05-01 2024-05-01 Unemployment Rate in Alaska 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:09-05:00 25 None
IAUR IAUR 2024-05-01 2024-05-01 Unemployment Rate in Iowa 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:09-05:00 25 None
UTUR UTUR 2024-05-01 2024-05-01 Unemployment Rate in Utah 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:12-05:00 24 None
PRUR PRUR 2024-05-01 2024-05-01 Unemployment Rate in Puerto Rico 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:12-05:00 20 None
INUR INUR 2024-05-01 2024-05-01 Unemployment Rate in Indiana 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:10-05:00 26 None
LAUR LAUR 2024-05-01 2024-05-01 Unemployment Rate in Louisiana 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:08-05:00 25 None
HIUR HIUR 2024-05-01 2024-05-01 Unemployment Rate in Hawaii 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:12-05:00 23 None
LASMT261982000000003 LASMT261982000000003 2024-05-01 2024-05-01 Unemployment Rate in Detroit-Warren-Dearborn, ... 1990-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-05-01 10:08:06-05:00 8 None
NDUR NDUR 2024-05-01 2024-05-01 Unemployment Rate in North Dakota 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:12-05:00 24 None
MTUR MTUR 2024-05-01 2024-05-01 Unemployment Rate in Montana 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:12-05:00 23 None
DCUR DCUR 2024-05-01 2024-05-01 Unemployment Rate in the District of Columbia 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:13-05:00 19 None
MSUR MSUR 2024-05-01 2024-05-01 Unemployment Rate in Mississippi 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:12-05:00 19 None
MEUR MEUR 2024-05-01 2024-05-01 Unemployment Rate in Maine 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:11-05:00 19 None
NEUR NEUR 2024-05-01 2024-05-01 Unemployment Rate in Nebraska 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:14-05:00 18 None
LASMT391746000000003 LASMT391746000000003 2024-05-01 2024-05-01 Unemployment Rate in Cleveland-Elyria, OH (MSA) 1990-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-05-01 10:08:08-05:00 12 None
CTUR CTUR 2024-05-01 2024-05-01 Unemployment Rate in Connecticut 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:14-05:00 19 None
IDUR IDUR 2024-05-01 2024-05-01 Unemployment Rate in Idaho 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:14-05:00 18 None
NHUR NHUR 2024-05-01 2024-05-01 Unemployment Rate in New Hampshire 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:15-05:00 17 None
RIUR RIUR 2024-05-01 2024-05-01 Unemployment Rate in Rhode Island 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:14-05:00 17 None
KSUR KSUR 2024-05-01 2024-05-01 Unemployment Rate in Kansas 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:15-05:00 12 None
WYUR WYUR 2024-05-01 2024-05-01 Unemployment Rate in Wyoming 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:13-05:00 17 None
DEUR DEUR 2024-05-01 2024-05-01 Unemployment Rate in Delaware 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:13-05:00 16 None
VTUR VTUR 2024-05-01 2024-05-01 Unemployment Rate in Vermont 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:14-05:00 15 None
SDUR SDUR 2024-05-01 2024-05-01 Unemployment Rate in South Dakota 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:15-05:00 13 None
CMWRUR CMWRUR 2024-05-01 2024-05-01 Unemployment Rate in Midwest Census Region 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:15-05:00 13 None
CSOUUR CSOUUR 2024-05-01 2024-05-01 Unemployment Rate in South Census Region 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:17-05:00 8 None
CNERUR CNERUR 2024-05-01 2024-05-01 Unemployment Rate in Northeast Census Region 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:18-05:00 4 None
CWSTUR CWSTUR 2024-05-01 2024-05-01 Unemployment Rate in West Census Region 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:18-05:00 6 None
CNEWUR CNEWUR 2024-05-01 2024-05-01 Unemployment Rate in New England Census Division 1976-01-01 2024-03-01 Monthly M Percent % Seasonally Adjusted SA 2024-04-22 10:41:16-05:00 6 None
In [ ]:
all_results = []    # stores al pandas dataframes results

for myid in unemp_df.index:
    results = fred.get_series(myid)
    results = results.to_frame(name=myid)
    #time.sleep(0.1)
    all_results.append(results)
In [ ]:
# concat all results
uemp_results = pd.concat(all_results, axis=1)
uemp_states = uemp_results.copy()
# Show if there are any NaN values
uemp_states.isna().sum(axis=1).plot(title='NaN data', lw=1)
Out[ ]:
<Axes: title={'center': 'NaN data'}>
No description has been provided for this image
In [ ]:
# drop NaN
uemp_states = uemp_states.dropna()
# states names
id_to_state = unemp_df['title'].str.replace('Unemployment Rate in ','').to_dict()
uemp_states.columns = [id_to_state[c] for c in uemp_states.columns]
px.line(uemp_states, title='Unemployment Rate by State')

4. Pull Last Month Unemployment by State¶

In [ ]:
# current date
current_date = datetime.now()
last_month_date = current_date - timedelta(days=35)
date = last_month_date.strftime('%Y-%m-01')
month = calendar.month_name[last_month_date.month]
title_date = f"{date[:4]}-{month.lower()}-{date[-2:]}"

ax = uemp_states.loc[uemp_states.index == date].T \
    .sort_values(date) \
    .plot(kind='barh', figsize=(8, 12), width=0.7, edgecolor='black',
          title='Unemployment Rate by State, ' + title_date)
ax.legend().remove()
ax.set_xlabel('% Unemployed')
plt.show()
No description has been provided for this image

5. Pull participation rate¶

In [ ]:
part_df = fred.search('participation rate state', filter=('frequency', 'Monthly'))
part_df = part_df.query('seasonal_adjustment == "Seasonally Adjusted" and units == "Percent"')
part_df = part_df.loc[part_df['title'].str.contains('Labor Force Participation Rate for ')]

part_all_results = []    # stores al pandas dataframes results

for myid in part_df.index:
    results = fred.get_series(myid)
    results = results.to_frame(name=myid)
    #time.sleep(0.1)
    part_all_results.append(results)

part_results = pd.concat(part_all_results, axis=1)
part_states = part_results.copy()

# drop NaN
part_states = part_states.dropna()

# states names
part_id_to_state = part_df['title'].str.replace('Labor Force Participation Rate for ','').to_dict()
part_states.columns = [part_id_to_state[c] for c in part_states.columns if c in part_id_to_state.keys()]
px.line(part_states, title='Participation Rate by State')

6. Plot Unemployment vs Participation for the last 4 years¶

In [ ]:
# Fix DC
uemp_states = uemp_states.rename(columns={'the District of Columbia':'District Of Columbia'})
year = current_date.year
year_gap = current_date.year - 4

fig, axs = plt.subplots(10, 5, figsize=(30, 30), sharex=True)
axs = axs.flatten()

i = 0
for state in uemp_states.columns:
    if state in ["District Of Columbia","Puerto Rico"]:
        continue
    if 'MSA' in state:
        continue
    if i < len(axs):          
        ax2 = axs[i].twinx() 
        uemp_states.query('index >= @year_gap and index < @year')[state] \
            .plot(ax=axs[i], label='Unemployment')
        part_states.query('index >= @year_gap and index < @year')[state] \
            .plot(ax=ax2, label='Participation', color=color_pal[1])
        ax2.grid(False)
        axs[i].set_title(state)
        i += 1
    else:
        break
plt.tight_layout()
plt.show()
No description has been provided for this image